Skip to content

YARN-11045. ATSv2 storage monitor fails to read from hbase cluster#3796

Merged
jojochuang merged 2 commits intoapache:trunkfrom
virajjasani:YARN-11045-trunk
Dec 15, 2021
Merged

YARN-11045. ATSv2 storage monitor fails to read from hbase cluster#3796
jojochuang merged 2 commits intoapache:trunkfrom
virajjasani:YARN-11045-trunk

Conversation

@virajjasani
Copy link
Contributor

@virajjasani virajjasani commented Dec 13, 2021 •
edited
Loading

Description of PR

HBase compatible guava dependency is bit messed up i.e. timelineservice-hbase modules are still being built with Hadoop's guava version (defined in hadoop-project) and this creates issues with HBaseStorageMonitor reading records from hbase cluster:

java.lang.RuntimeException: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError: com.google.common.net.HostAndPort.getHostText()Ljava/lang/String;
        at org.apache.hadoop.hbase.client.AbstractClientScanner$1.hasNext(AbstractClientScanner.java:95)
        at org.apache.hadoop.yarn.server.timelineservice.storage.reader.TimelineEntityReader.readEntities(TimelineEntityReader.java:283)
        at org.apache.hadoop.yarn.server.timelineservice.storage.HBaseStorageMonitor.healthCheck(HBaseStorageMonitor.java:77)
        at org.apache.hadoop.yarn.server.timelineservice.storage.TimelineStorageMonitor$MonitorThread.run(TimelineStorageMonitor.java:89)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError: com.google.common.net.HostAndPort.getHostText()Ljava/lang/String;
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:260)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:233)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:394)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:368)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:143)
        at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
        ... 3 more
Caused by: java.lang.NoSuchMethodError: com.google.common.net.HostAndPort.getHostText()Ljava/lang/String;
        at org.apache.hadoop.hbase.net.Address.getHostName(Address.java:72)
        at org.apache.hadoop.hbase.net.Address.toSocketAddress(Address.java:57)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:576)
        at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:37250)
        at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:405)
        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:274)
        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:231)

TimelineReaderServer logs after fixing the issue:

Screenshot 2021-12-14 at 11 16 19 PM

How was this patch tested?

  1. Tested locally
  2. Verified guava jar present in timelineservice/lib
  3. Run all hbase UTs
  4. Checked timelinereader logs to confirm the storage monitor is not failing

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?

@virajjasani
Copy link
Contributor Author

virajjasani commented Dec 13, 2021 •
edited
Loading

@aajisaka @jojochuang @tasanuma Could you please take a look?

@hadoop-yetus
Copy link

hadoop-yetus commented Dec 14, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 43s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 shelldocs 0m 1s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 13m 20s Maven dependency ordering for branch
+1 💚 mvninstall 21m 34s trunk passed
+1 💚 compile 9m 41s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 8m 24s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 mvnsite 8m 1s trunk passed
+1 💚 javadoc 4m 58s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌ javadoc 0m 16s /branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-1-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-1 in trunk failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌ javadoc 0m 17s /branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-2-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-2 in trunk failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
+1 💚 shadedclient 28m 42s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 28s Maven dependency ordering for patch
+1 💚 mvninstall 6m 23s the patch passed
+1 💚 compile 9m 4s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 9m 4s the patch passed
+1 💚 compile 8m 14s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 8m 14s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 6m 26s the patch passed
+1 💚 shellcheck 0m 23s No new issues.
+1 💚 xml 0m 6s The patch has no ill-formed XML file.
+1 💚 javadoc 4m 29s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌ javadoc 0m 11s /patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-1-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-1 in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌ javadoc 0m 12s /patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-2-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-2 in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
+1 💚 shadedclient 28m 3s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 230m 39s /patch-unit-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn in the patch passed.
+1 💚 unit 1m 5s hadoop-yarn-server-timelineservice-hbase-common in the patch passed.
+1 💚 unit 1m 0s hadoop-yarn-server-timelineservice-hbase-client in the patch passed.
+1 💚 unit 0m 44s hadoop-yarn-server-timelineservice-hbase-server-1 in the patch passed.
+1 💚 unit 0m 52s hadoop-yarn-server-timelineservice-hbase-server-2 in the patch passed.
+1 💚 asflicense 1m 2s The patch does not generate ASF License warnings.
386m 57s
Reason Tests
Failed junit tests hadoop.yarn.csi.client.TestCsiClient
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3796/1/artifact/out/Dockerfile
GITHUB PR #3796
Optional Tests dupname asflicense codespell shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml
uname Linux d9af839a777b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 1c6c36e
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3796/1/testReport/
Max. process+thread count 952 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-1 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-2 U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3796/1/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@virajjasani
Copy link
Contributor Author

virajjasani commented Dec 14, 2021

Javadoc warnings are not relevant to this PR. They can be taken up in a separate task to fix existing checkstyle issues.

jojochuang
jojochuang approved these changes Dec 14, 2021
View reviewed changes
Copy link
Contributor

@jojochuang jojochuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks a lot for reporting the bug and providing the fix!

@jojochuang
Copy link
Contributor

jojochuang commented Dec 14, 2021

Will merge later today

@minni31
Copy link
Contributor

minni31 commented Dec 14, 2021

@virajjasani , I tried porting your changes and I can still see yarn resourcemanager having the same issue for me.

@virajjasani
Copy link
Contributor Author

virajjasani commented Dec 14, 2021 •
edited
Loading

@minni31 have you also done changes in bin/yarn as per this PR?

After porting the changes, building the tarball (mvn clean package -Pdist -DskipTests -Dtar -Dmaven.javadoc.skip=true) adds guava-12.0 jar under share/hadoop/yarn/timelineservice/lib.
And as per bin/yarn change, starting timeline reader server will always add timelineservice/lib/* contents first, overriding hadoop's own guava-27.0 jar version in the classpath. Can you please confirm if this works for you?

@minni31
Copy link
Contributor

minni31 commented Dec 14, 2021

Yes @virajjasani, it fails, even after changes in bin/yarn

@virajjasani
Copy link
Contributor Author

virajjasani commented Dec 14, 2021

With the changes on this PR, it works for me.
From the timeline reader logs, can you provide classpath value? It must be at the beginning (3rd or 4th line from the beginning). Does the classpath have guava 12.0 jar? (guava 12.0 jar should be present before guava 27.0 jar)

For me, timeline reader works well with this patch. Without this patch, the Exception is thrown as mentioned on the PR description.
Let's try to understand what might be missing in your test. Which platform are you using and which command?
I tried on Mac and Linux both with command bin/yarn --daemon start timelinereader

@virajjasani
Copy link
Contributor Author

virajjasani commented Dec 14, 2021

Just pushed an addendum to make the same Yarn classpath change for Windows OS.

@virajjasani
Copy link
Contributor Author

virajjasani commented Dec 14, 2021

I had an offline sync with @minni31
The recent commit on this PR helps for TimelineReaderServer on Windows platform. However, there is an additional issue related to RM and NM (YARN-11047). I will create PR for the issue once this is merged since both changes are inter-related.
Thanks @minni31

FYI @jojochuang

@hadoop-yetus
Copy link

hadoop-yetus commented Dec 14, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 42s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 shelldocs 0m 1s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 13m 5s Maven dependency ordering for branch
+1 💚 mvninstall 21m 39s trunk passed
+1 💚 compile 9m 42s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 8m 25s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 mvnsite 8m 2s trunk passed
+1 💚 javadoc 5m 0s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌ javadoc 0m 17s /branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-1-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-1 in trunk failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌ javadoc 0m 17s /branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-2-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-2 in trunk failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
+1 💚 shadedclient 28m 17s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 27s Maven dependency ordering for patch
+1 💚 mvninstall 6m 31s the patch passed
+1 💚 compile 9m 0s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 9m 0s the patch passed
+1 💚 compile 8m 20s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 8m 20s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 6m 25s the patch passed
+1 💚 shellcheck 0m 23s No new issues.
+1 💚 xml 0m 5s The patch has no ill-formed XML file.
+1 💚 javadoc 4m 24s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌ javadoc 0m 11s /patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-1-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-1 in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌ javadoc 0m 10s /patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-2-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-2 in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
+1 💚 shadedclient 28m 20s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 230m 40s /patch-unit-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn in the patch passed.
+1 💚 unit 0m 52s hadoop-yarn-server-timelineservice-hbase-common in the patch passed.
+1 💚 unit 0m 52s hadoop-yarn-server-timelineservice-hbase-client in the patch passed.
+1 💚 unit 0m 42s hadoop-yarn-server-timelineservice-hbase-server-1 in the patch passed.
+1 💚 unit 0m 44s hadoop-yarn-server-timelineservice-hbase-server-2 in the patch passed.
+1 💚 asflicense 1m 2s The patch does not generate ASF License warnings.
386m 24s
Reason Tests
Failed junit tests hadoop.yarn.csi.client.TestCsiClient
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3796/2/artifact/out/Dockerfile
GITHUB PR #3796
Optional Tests dupname asflicense codespell shellcheck shelldocs mvnsite unit compile javac javadoc mvninstall shadedclient xml
uname Linux e31ff50d5f08 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 86b25f3
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3796/2/testReport/
Max. process+thread count 949 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-1 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-2 U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3796/2/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

hadoop-yetus commented Dec 15, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 43s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 shelldocs 0m 1s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 12m 53s Maven dependency ordering for branch
+1 💚 mvninstall 21m 43s trunk passed
+1 💚 compile 9m 39s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 8m 21s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 mvnsite 8m 0s trunk passed
+1 💚 javadoc 4m 58s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌ javadoc 0m 16s /branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-1-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-1 in trunk failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌ javadoc 0m 17s /branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-2-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-2 in trunk failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
+1 💚 shadedclient 28m 24s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 28s Maven dependency ordering for patch
+1 💚 mvninstall 6m 25s the patch passed
+1 💚 compile 8m 58s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 8m 58s the patch passed
+1 💚 compile 8m 16s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 8m 16s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 6m 22s the patch passed
+1 💚 shellcheck 0m 24s No new issues.
+1 💚 xml 0m 5s The patch has no ill-formed XML file.
+1 💚 javadoc 4m 29s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌ javadoc 0m 12s /patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-1-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-1 in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌ javadoc 0m 11s /patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-server_hadoop-yarn-server-timelineservice-hbase-server-2-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt hadoop-yarn-server-timelineservice-hbase-server-2 in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
+1 💚 shadedclient 29m 4s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 231m 55s /patch-unit-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn in the patch passed.
+1 💚 unit 1m 6s hadoop-yarn-server-timelineservice-hbase-common in the patch passed.
+1 💚 unit 1m 1s hadoop-yarn-server-timelineservice-hbase-client in the patch passed.
+1 💚 unit 0m 43s hadoop-yarn-server-timelineservice-hbase-server-1 in the patch passed.
+1 💚 unit 0m 52s hadoop-yarn-server-timelineservice-hbase-server-2 in the patch passed.
+1 💚 asflicense 1m 2s The patch does not generate ASF License warnings.
388m 25s
Reason Tests
Failed junit tests hadoop.yarn.csi.client.TestCsiClient
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3796/3/artifact/out/Dockerfile
GITHUB PR #3796
Optional Tests dupname asflicense codespell shellcheck shelldocs mvnsite unit compile javac javadoc mvninstall shadedclient xml
uname Linux b8a6b50ac3e7 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 86b25f3
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3796/3/testReport/
Max. process+thread count 946 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-1 hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-server/hadoop-yarn-server-timelineservice-hbase-server-2 U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3796/3/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@minni31
Copy link
Contributor

minni31 commented Dec 15, 2021

Thanks @virajjasani
This PR is fixing the issue in yarn timelinereader and issues in NM and RM can be handled as part of YARN-11047.

@jojochuang
Copy link
Contributor

jojochuang commented Dec 15, 2021

+1 based on the good faith of Viraj. Merging it now.

@jojochuang jojochuang merged commit f43ac31 into apache:trunk Dec 15, 2021
hotcodemacha pushed a commit to hotcodemacha/hadoop that referenced this pull request Dec 22, 2021
@virajjasani @hotcodemacha
YARN-11045. ATSv2 storage monitor fails to read from hbase cluster (a…
2d1142a 
…pache#3796)
HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
@virajjasani
YARN-11045. ATSv2 storage monitor fails to read from hbase cluster (a…
c373a7f 
…pache#3796)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jojochuang jojochuang jojochuang approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@virajjasani @hadoop-yetus @jojochuang @minni31